kolmogorov-arnold convolution
KAConvText: Novel Approach to Burmese Sentence Classification using Kolmogorov-Arnold Convolution
Thu, Ye Kyaw, Aung, Thura, Oo, Thazin Myint, Supnithi, Thepchai
This paper presents the first application of Kolmogorov-Arnold Convolution for Text (KAConvText) in sentence classification, addressing three tasks: imbalanced binary hate speech detection, balanced multiclass news classification, and imbalanced multiclass ethnic language identification. We investigate various embedding configurations, comparing random to fastText embeddings in both static and fine-tuned settings, with embedding dimensions of 100 and 300 using CBOW and Skip-gram models. Baselines include standard CNNs and CNNs augmented with a Kolmogorov-Arnold Network (CNN-KAN). In addition, we investigated KAConvText with different classification heads - MLP and KAN, where using KAN head supports enhanced interpretability. Results show that KAConvText-MLP with fine-tuned fastText embeddings achieves the best performance of 91.23% accuracy (F1-score = 0.9109) for hate speech detection, 92.66% accuracy (F1-score = 0.9267) for news classification, and 99.82% accuracy (F1-score = 0.9982) for language identification.
Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies
The rapid evolution of deep learning architectures has significantly advanced the field of computer vision, particularly in tasks that require the analysis of complex spatial data. Convolutional Neural Networks (CNNs), initially proposed by LeCun et al. [1], have become a cornerstone in this domain due to their ability to efficiently process highdimensional data arrays such as images. These networks typically employ linear transformations followed by activation functions in their convolutional layers to discern spatial relationships, thereby reducing the number of parameters needed to capture intricate patterns in visual data. Since 2012, following the success of AlexNet [2] in the ImageNet classification challenge, CNNs have dominated the field of computer vision until the emergence of Vision Transformers [3]. Innovations such as Residual Networks [4] and Densely Connected networks [5], along with numerous subsequent works, have significantly advanced the achievable quality of models based on convolutional layers, enabling the effective training of very large and deep networks. In segmentation tasks, especially within the biomedical domain, CNNs have also become foundational with the advent of the U-Net [6] architecture, which has subsequently inspired a whole family of U-Net-like architectures for segmentation tasks. Recent developments in deep learning have seen the integration of sophisticated mathematical theories into neural network architectures, enhancing their capability to handle complex data structures. One such innovation is the Kolmogorov-Arnold Network (KAN) [7], which leverages the Kolmogorov-Arnold theorem to incorporate splines into its architecture, offering a compelling alternative to traditional Multi-Layer Perceptrons (MLPs).